Two studies of a human-AI collaborative design tool were carried out in order to understand the influence design recommendations have on the design process. The tool investigated is based on an evolutionary algorithm attempting to design a virtual car to travel as far as possible in a fixed time. Participants were able to design their own cars, make recommendations to the algorithm and view sets of recommendations from the algorithm. The algorithm-recommended sets were designs which had been previously tested; some sets were simply randomly picked and other sets were picked using MAP-Elites. In the first study 808 design sessions were recorded as part of a science outreach program, each with analytical data of how each participant used the tool. To provide context to this quantitative data, a smaller double-blind lab study was also carried out with 12 participants. In the lab study the same quantitative data from the large scale study was collected alongside responses to interview questions. Although there is some evidence that the MAP-Elites provide higher-quality individual recommendations, neither study provides convincing evidence that these recommendations have a more positive influence on the design process than simply a random selection of designs. In fact, it seems that providing a combination of MAP-Elites and randomly selected recommendations is beneficial to the process. Furthermore, simply viewing recommendations from the MAP-Elites had a positive influence on engagement in the design task and the quality of the final design produced. Our findings are significant both for researchers designing new mixed-initiative tools, and those who wish to evaluate existing tools. Most significantly, we found that metrics researchers currently use to evaluate the success of human-AI collaborative algorithms do not measure the full influence these algorithms have on the design process.

Research on mixed-initiative interaction and assistance is still in its infancy but is poised to blossom into a wellspring of innovation that promise to change the way we work with computing systems--and the way that computing systems work with us. I share reflections about the opportunities ahead for developing computational systems with the ability to engage people in a deeply collaborative manner, founded on their ability to support fluid mixed-initiative problem solving. Such collaborative intelligence sits at the veritable heart of human civilization. In the course of daily life, we assume and rely on a rich interleaving of efforts to achieve goals while immersed in shared context. We continue to engage one another in efficient, tightly woven collaborations, reasoning with remarkable efficiency about the beliefs, preferences, intentions, and skills of potential collaborators. The inferences underlying successful collaborations typically stream in such an effortless and subconscious manner that we often fail to recognize the elegance and sophistication of these capabilities. The magic of human collaborative competency comes to the foreground with attempts to extend these skills to computational systems. Developing a better understanding of the core aspects of intelligence that enable people to collaborate with fluidity promises to enable new kinds of human-computer collaboration. The nascent area of research on mixed-initiative interaction centers on developing methods that enable computing systems to support an efficient, natural interleaving of contributions by people and computers, aimed at converging on solutions to problems. In mixed-initiative interaction, people and computers take initiatives to contribute to solving a problem, achieving a goal, or coming to a joint understanding. Conversational dialogue is an oft-cited example of mixed-initiative interaction, referring to the ability of each participant in a dialogue to take initiative to guide or add to a discussion. Endowing an automated dialogue system with the ability both to take initiative ("What city do you wish a flight to?") and to allow people to take conversational initiative ("Wait, I'd like to add a side trip.") However, mixed-initiative interaction extends beyond spoken conversations to include a broad spectrum of collaborative problem solving marked by an interleaving of contributions by different participants. Mastering mixed-initiative interaction poses a constellation of fascinating challenges and opportunities for AI researchers. Figure 1 highlights the core challenge of seeking mutual understanding or grounding of joint activity. Joint activity describes the behavior displayed by people working together to solve a mutual goal.

Due to the increasing importance of service offerings as a revenue source and increasing competition among service providers, it is important for companies to optimize both the customer experience as well as the associated cost of providing the service. This article describes a mixed-initiative system that was created to improve customer support for problems customers encountered with their appliances. The mixed-initiative system improved the correctness of the diagnostic process, the speed of the process, and user satisfaction. The tool has been in use since 1999 and has provided more than $50 million in financial benefits by increasing the percentage of questions that could be answered without sending a field service technician to the customers' homes. These systems are rather popular with companies because they save money--the companies' money, that is.

Mixed-initiative systems are a popular approach to building intelligent systems that can collaborate naturally and effectively with people. But true collaborative behavior requires an agent to possess a number of capabilities, including reasoning, communication, planning, execution, and learning. We describe an integrated approach to the design and implementation of a collaborative problem-solving assistant based on a formal theory of joint activity and a declarative representation of tasks. This approach builds on prior work by us and by others on mixed-initiative dialogue and planning systems.

Customer service is defined as the ability of a company to afford the service requestor with the expressed need. Due to the increasing importance of service offerings as a revenue source and increasing competition among service providers, it is important for companies to optimize both the customer experience as well as the associated cost of providing the service. For more complex interactions with higher value, mixed-initiative systems provide an avenue that gives a good balance between the two goals. This article describes a mixed-initiative system that was created to improve customer support for problems customers encountered with their appliances. The tool helped call takers solve customers' problems by suggesting questions aiding the diagnosis of these problems. The mixed-initiative system improved the correctness of the diagnostic process, the speed of the process, and user satisfaction. The tool has been in use since 1999 and has provided more than $50 million in financial benefits by increasing the percentage of questions that could be answered without sending a field service technician to the customers' homes. Another mixed-initiative tool, for answering e-mail from customers, was created in 2000.

Mixed-initiative systems are a popular approach to building intelligent systems that can collaborate naturally and effectively with people. But true collaborative behavior requires an agent to possess a number of capabilities, including reasoning, communication, planning, execution, and learning. We describe an integrated approach to the design and implementation of a collaborative problem-solving assistant based on a formal theory of joint activity and a declarative representation of tasks. This approach builds on prior work by us and by others on mixed-initiative dialogue and planning systems.

Conversational dialogue is an oft-cited example of mixed-initiative interaction, referring to the ability of each participant in a dialogue to take initiative to guide or add to a discussion. Endowing an automated dialogue system communicate, and coordinate with with the ability both to take initiative ("What In the course like to add a side trip.") However, of efforts to achieve goals while immersed mixed-initiative interaction extends beyond in shared context. We continue to engage spoken conversations to include a broad spectrum one another in efficient, tightly woven of collaborative problem solving marked collaborations, reasoning with remarkable efficiency by an interleaving of contributions by different about the beliefs, preferences, intentions, participants. Mastering mixed-initiative interaction poses The inferences underlying successful collaborations a constellation of fascinating challenges and typically stream in such an effortless opportunities for AI researchers.